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ABSTRACT 

^ This paper has been prepared to assist the United 

States Department of Labor to explore new approaches to evaluating 
and measuring the performance of employment and training activities 
for youth. As one of several tools for evaluating success of local 
youth training programs, "benchmarking" provides a system for 
measuring the development of those trained with Comprehensive 
Employment and Training Act (CETA) funds. The paper consists of five 
sections. Section 1 contains definitions and a discussion of the 
terms and concepts involved in any consideration of benchmarking. 
Section 2 provides a historical overview of the development of 
competency measures in the secondary education and vocational 
education systems, along with the Consolidated Youth Employment 
Program ^{CYEP) experience with benchmarking. In section 3, a number 
of issued that should be considered before a benchmarking system is 
designed are discussed, and the importance of developing benchmarks 
in conjunction with local employers is emphasized. Section 4 focuses 
on the problems that could be encountered in implementing a 
benchmarking system. The final section contains a brief listing of 
the qualities one would expect to find in an exemplary benchmarking 
system. (A large portion of the research on which this paper is based 
was generated as part of the "knowledge development" program of the 
recent youth experimental and demonstration efforts.) (KC) 
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Foreword 



This paper has been prepared to assist the United States Department of 
Labor to explore new approaches to evaluating and measuring the perform- 
ance of employment anctjbrainin activities for youth. The development 
of new or improved perrormance standards for youth programs is being 
conducted in much the same fashion as the recent work to develop per- 
formance standards for adult programs* The work is being undertaken 
cooperatively by the Employment and Training Administration's (ETA) 
Office of Youth Programs and Office of Performance Management. 

In formulating standards for CETA youth programs, ETA is utilizing the 
expertise of a wide range of interested parties. In particular, the 
Department has asked two groups of people actively involved in the 
employment and training system to reflect on the needs of the system, 
the past and present evaluation practices for "youth" programs, and on 
their own personal experiences before making recommendations on per- 
formance measures for youth programs. An Advisory Committee (AC) makes 
policy recommendations and provides overall direction to the work, 
while a smaller Technical' Work Group (TWG) drafts papers defining 
issues and approaches for Advisory Committee review and consideration. 
The TWG has day-to-day responsibility for the development of performance 
standards; the AC provides guidance to the TWG. 

At present, these two groups consist of the following representatives: 



•Technical Work GrouD (TWG) 


Advisory Corrmittee (AC) 


8 Prime SponsoV staff 


15 Prinie Soonsors 


(Oirecxors or their Associates) 


5 National Office (dSl) staff 


5 National Office Administrators 


2 Regional Office (OOL) staff 


2 Regional Administrators 


1 Local Educational Agency staff 


3 Public Interest Grouos 




1 Local Educational Agency Director 




1 National Education Administrator 



A. L. Neilum and Associates has been asked to provide research and 
policy development assistance to the Work Group. As part of this 
assistance, in late Septernber ETA asked ALNA to produce a short paper 
on benchmarking. As a reference for the TWG, this paper will summarize 
the readily-available benchmarking literature, and serve as a stepping- 
off point for future actions by ETA to promulgate and implement new 
performance standards requirements developed by the Work Group and 
Advisory Group. 
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ALNA staff has analyzed the literature recommended by members of the 
Work Group as well as other related dc uments available through the DOL 
Resource Centers, the Massachusetts Regional Vocational Education 
Curriculum Resource Center and our own files and resource materials. 
Included i,n this literature search were several products of the knowl- 
edge development effort conducted as part of the Youth Employment 
Demonstr^itiDn Projects Act (YEDPA). Of particular assistance 'to us 
were Progress Toward Integrating Services for Youth , a series of quar- 
terly reports prepared by the Technical Assistance antj Training Corpora- 
tion wKich evaluates the progress of the Consolidated Youth Employment 
Program (CYEP); and a three-volume report prepared by the Syracuse Re- 
search Corporation on Benchmarking: A State-of-the-Art Review . Although 
this search has not necessarily uncovered all of the relevant literature 
on benchmarking, it does include the major current documentation on 
benchmarking efforts in the employment and training system as well as a 
considerable portion of the recent educational literature on benchmarking. 
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Introduction 



/ 



. ^^^^^^Jgnchmarking is an important concept for the employment and train- 
ing system • Successful implementation of comprehensive benchmarking 
systems could help employment and training programs provide clients 
with more useful service, linK training programs to local employers and 
more carefully manage the services provided. To date, only preliminary 

efforts have been made to implement benchmarking systems in the employ- 
ee 

ment and training system. Although these efforts have not been entirely 
successful, they have provided us with a great deal of useful guidance 
for future efforts to develop benchmarking systems. 

Section I of this paper contains definitions and a discussion of 
the terms and concepts involved in any consideration of benchmarking. 
In addition to explaining competency indicators, competency areas and 
benchmarks, this section explains how these can combine with an assess- 
' ment system to form a comprehensive benchmarking system. 

Section II provides an historical overview of the^ development of 
competency measures in the secondary education and vocational education 
systems. In addition, the CYEP experience with benchmarking is dis- 
cussed. This experience provides us with considerable insight into how 
future efforts to construct benchmarking systems ought to be conducted. 

In Section III, we discuss a number of issues which ought to be 
considered before designing a benchmarking system. This section stresses 
the need to carefully think about the conceptual framework for a bench-, 
marking system before designing it. In addition, the importance of 
developing benchmarks in conjunction with local employers is emphasized. 

Section IV focuses on the' problems which could be encountered in 
actually implementing a benchmarking system. In this section we argue 
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that ihe implenentation of benchmarking systems is truly beneficial only 
if sufficient time and resources are allocated to allow the system to 
function properly. 

" The Findings Section which concludes this report contains a brief 
listing of the qualities one would expect to find in an exemplary bench- 
marking system. The list is not all inclusive. Rather, it is an attempt 
to summarize the most prominent features • 

A large portion of the research on which this report is based was 
generated as, part of the "Knowledge Development" program of the recent 
youth experimental and demonstration efforts. Of particular assistance 
were the Syracuse Research Corporation series Benchmarking: A State-of- 
the-Art Review, and the Quarterly Progress Reports on the Consolidated 
Youth Employment Project which were prepared by the Technical Assistance 
and Training Corporation, We have also benefited from the wise counsel 
of many of the Youth Performance Standards Technical Work Group members, 
and from the support of the ETA Office of Youth Programs and Office of 
Performance Management, 

Perhaps the most encouraging aspect of preparing this report was the 
positive feedback engendered during our discussions with those engaged in 
benchmarking efforts. Despite some initial difficulty in implementing 
benchmarking systems, there were CETA prime sponsors who have develbped 
and implemented them successfully in their programs. 
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1. Defining the Concept ) . ^ 

\ As one of several tools for evaluating the success of youth employ- 
ment and training strategies at the local level, benchmarking provides 
dn explicit and formal system for measuring and documenting' the employ- 
ability development of CETA participants. In a general context, CETA 
benchmarks "can be defined as standards of performance or achievement 
which are perceived as necessary to obtain and retain unsubsidized em- 
ployjnent* " • 

"Benchmarks" is only one label that could be given to standards 
against which to measure participant progress. Some critics argue that 
the use of the term does little but confuse the intent^, and that 
"benchmarks" is just a synonym for other terms which ar6 more current 
and commonly understood— 1 ike performance standards, minimum competen- 
cies,, achievement standards, measurable outcomes. But discussions with 
prime sponsors and program operators who have implemented benchmarking 
systems suggest thdt the term should be retained. In their report, 
Benchmarking: A State-of-the-Art Review » Syracuse Research Corporation 
provides a general argument* for the unique use of the term: 

Benchmarks are not to be confused with performance standards 
as the CETA system currently knows them, nor with assessment 
y as the system currently practices it. Nor are benchmarks 
exactly like minimum competencies in education circles, since 
they neither exclusively apply to basic educational competen- 
cies, nor -would the "minimum competencies" for different par- 
ticipants in a benchmarking system necessarily be the. same. 2 

CETA has used the benchmarking concept informally whenever a pro- 
gram operator, counselor, teacher, trainer, or work supervisor calls 
upon some standard against which to measure performance of a CETA par- 
ticipant. When prime sponsors require participants to have a high school 
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diploma or achieve a specific educational level, they have established 

a benchmark. When a worksite supervisor requires trainees to report to 

work on time or when a trainee's work attitude is evaluated according 

to some set of criteria, benchmarks have been applied. However, the 

development of those measures, the methods for assessing performance 

and the documentation of individual participant achievements, have in 

n?iny cases resulted from informal processes and inconsistent application 
3 

of procedures. The current use of the benchmarking concept for youth 
employment and training programs is innovative in that it recogniz.es the 
need to move from implicit and informal formulation and use to explicit 
and formal statement and application. 

Benchmarking, in an employment and training context, is a process 
for linking the demands of the local job market with the employabil ity 
development needs of CETA participants. Benchmarking indicates how and 
to what extent those needs can be met' through the provision of CETA ser- 
vices or a combination of CETA and non-CETA services, CETA benchmarks 
for youth programs \ould be defined'^as performance standards that 
indicate the levels of ability and knowledge which a youth needs to 
attain in order to compete in the local labor market. Achieving bench- 
marks could thus indicate a youth's ability to succeed in a job. 

'Although the term "l)enchmarking" refers to a single process, 
i.e., the measurement of performance according to some set of stan- 
dards, a benchmarking system is actually comprised of four related 
elements. An example of this relationship is contained in Figure 1: 
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Figure 1 








CompeUincj /<rea 


Competency 
ImBcntor 


Benchmark 


Assessment 


^Woffe Maturity 
• 

0 


* * 

Participant will 
demonstrate 
good attendance. 


by showing, up on 
time nirfg^clays 
out of ten on a 
work^experience 
placement. . . 

1 


as measured 
by employer 
attendance 
reports • 



A. Competency Areas 

Employment and training competency areas can be defined as knowledge 
or skill areas that are perceived as necessary for an individual to 
obtain a job. Competency areas define general behaviors that partici- 
pants are expected to demonstrate. They result 'from the question, 
"What skills, knowledge and attitudes can we identify, through prior 
experience with CETA youth programs, to be most relevant and impojjtant 
to a youth's ability to compete in the labor market?" 

In determining appropriate needs for youth employability develop- 
ment, competencies should refer to more than occupational skills or 
qualifications to obtain a job. Surveys of employer expectations of 
job applicants indicate interest in individuals who can demonstrate 
compiatence in such areas as work attitude, punctuality, ability to 
follow instructions, as well as demonstrated levels of competency in 
reading and math.* Sec^tion II C of this paper will discuss the kinds 
of information acquired through youth knowledge development activities 
(specifically VEDP and CYEP) that allow us to make certain informed 
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statements about youth employability development needs. Based on this 
information, it seems that appropriate competency areas for youth would 
include work maturity, educational,, occupational, and pre-employment 
skill.^. These competency areas will be,, discussed in .detail later in 
this paper. 

B. Competency Indicators 

Competency Indicators reflect the devefopment of a competency in 
an applied coritext. While competency areas identify general behaviors 
and areas of skill or knowledge, competency indicators break each of 
those Areas into activities (e.g., written tests, mev^hanical tests, 
performance on the job, etc.) which measure an individual's proficiency 
in performing a specific task. Competency indicators result rrom the 
question, '-''What behaviors will indicate that a youth has attained com- 
petency 'in a given area?" 

C Benchmarks 

Benchmarks are closely related to competency indicators. While the 

latter refers to levels of skills or knowledge to be demonstrated, 

benchm&rj<s refer to performance standards at which a particular indivi- 

dual is expected to demonstrate competency. Benchmarks are established 

at the local level on the basis of local needs and capabilities to meet 
* 

those needs. Specific employment and training needs may vary with each 
prime sponsor, depending upon such variables as local labor market con- 

4 

ditions, the needs of the^^ervice population and the size and available 
resources of the prime sponsor delivery system. Benchmarks should 
reflect prime .sponsor reality— ^'Are we asking participants to demonstrate 
skills,' knowledge and attitudes that' will get them a job and can we 
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provide sufficient services to assist in achieving those competencies?" 
In addition, since a given benchmark is not necessarily equally relevant 
to all people, nor does everyone independently establish the same bench- 
, marks in their lifetime, benchmarks should be established at several 
proficiency levels that reflect key steps to attaining entry-level job 
requirements* 

D. ""Assessment 

Competency-building and benchmarking rely heavily on the assessment 
process to determine each youth's progress in attaining appropria1;e em- 
ployability skills. The assessment process should determine each 
youth's strengths and weaknesses relative to basic competencies and the 
capability, of the program to meet individual needs. Factors that should 
be addressed during assessment include a youth's interests and apti- 
tudes, current abilities and previous educational and work experiences 
as well as the additional skills needed to'compete s^uccessfully in the 
labor market. Major elements of an assessment process include the 
following: 

1. measuring the youth's initial level of employabil ity 
development in order to determine present levels of 
competency, 

2. providing a ?ationale for and identifying tentative 
long-range employabil ity development goals and com- 
petency .objectives for each youth, 

3. selecting the mix of services in'^which youth will 
participate in order to achieve the stated competency 
objectives, ' ' * 

4. measuring each youth's gains in competency areas 
and progress towards employabil ity development. 
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The assessment process should occur periodically, perhaps every 
quarter, or ideally upon completion of each unit of service, A youth* 
progress throughout the CETA program can be measured in various ways: 
(1) standardized tests for math and reading levels, (2) employer and 
worksite supervisor evaluations, (3) mechanical aptitude tests, 

(4) exercises that indicate understanding of specific skills, or 

(5) records of completion of a specified activity, like an attendance 
report • Reassessment involves feedback to the youth and leads to revi 
sion of a youth's individual goals and designation of services. The 
outcomes of these various types of assessment are documented in the 
client's Employabil ity Development Plan (EDP)r 

In summary, the basic elements of a benchmarking system can be 
defined in the following manner: 



1. Competency Areas (educational, occupational 
skills or other areas of measurable behavior) 
which are perceived as important work-related 
knowledge that must be acquired if youth are 
to move successfully into the labor market; 

2. Competency Indicators which are specific 
behaviors within a competency area which de- 
fine the level of proficiency which a parti- 
cipant has obtained; 

3. Benchmarks which are measurements of the level 
of competency attained by an individual par- 
ticipant in a specific competency area; and 

4. Assessment and Reassessment which are processes 
for determining an individual's progress in 
achieving basic competency levels. 
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II. The Development of Competency Measures in the 
Educational and CE TA Systems 

In this section, we will briefly trace the development of competency 
measures and benchmarking systems in the educational and CETA systems. The 
"mainstream" education system has been engaged in benchmarking-type activi- 
ties for much longer than the CETA system. The processes involved and sys- 
tems used by the education system can provide useful models for the CETA 
system as it reviews past benchmarking efforts in employment and training 
demonstrations like CYEP and VEDP and considers continuation of the bench- 
marking efforts for CETA youth activities. 

A. The Education System 

Most of the early efforts at mass education in the United States 
focused on providing basic skills in reading, writing and mathematics. As 
the affluence of the general population increased, people demanded more 
sophisticated educational programs from the public schools. In a sense, 
the goal of the public education system changed over time from merely pro- 
ducing individuals who possessed minimal literacy skills to developing a 
more "well-rounded" graduate. This well-rounded graduate possessed skills 
in the "three R's" as well as some knowledge of civics, history, geography, 
science, music, theatre, athletics, and the like. 

For reasons beyond the scope of this paper, some critics have argued 
that this move past the earlier focus on the "three R's" has had a detri- 
mental effect on the quality of the basic education offered to public school 
students. While attempting to produce students who were more broadly edu- 
cated in the arts and sciences, critics argued that public schools were 
turning out students who couldn't read, write or solve simple problems in 
math. Tnese criticisms have, in many cases, arisen at the same time as 
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criticisms that the public schools were too costly to run; that they were 
producing students without sufficient skills to successfully fill avail- 
able jobs; that secondary schools were sending off to colleges and uni- 
versities students who require a first-year program of remedial education; 
and that school systems across the country vary too widely in educational 
quality, 

A 

In response to these criticisms, many school systems have renewed the 
emphasis on basic skills and developed (or are in the process of developing) 
standardized, objective measures of "competence" to be applied to potential 
grade completers and^ graduates. It is not uncommon for local school dis- 
tricts to have competency-based approaches both for promoting students from 
one grade level to another and as a criterion for graduation. 

In a recent article in Phi D elta Kappan , a journal of higher education, 

the minimum competency-testing approach was defined in terms of three 

5 

general characteristics: 

1. All or most students of designated grades take paper-and- 
pencil tests to measure basic academic skills, life or 
survival skills, or functional literacy; 

2. A passing score or standard for acceptable levels of 
student performance is established; and 

3. Test results may be used to certify students for grade 
promotion, graduation, or diploma award; to classify 
students for or to place students in remedial or other 
special services; to allocate compensatory funds to 
districts; to evaluate or to certify schools or school 
di stricts ; or to eval uate teachers • 

W. James Popham, in another Phi Delta Kappan article, suggested seven 

identifiable features to characterize a "high quality" minimum competency 
6 

testing program. 

1. Constituency-Selected Competencies - All concerned con- 
stituencies are included in selecting competencies for 
testing. Those included in this process would be school 
boards, administrators, teachers and employers. 
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2. Appropriate Competency Tests - The tests are carefully constructed 
criterion-referenced instrunients accompanied by clear descriptions 
of the competencies they measured. The term "criterion-reference" 
refers to some indicator of performance that can be measured on 
the job or in a training situation (measurable behaviors). 

3. Teaching/Testing Congruence - The instructional prog^^am directly 
teaches the competencies measured by the tests. 

4. Multiple Testing Cpportunities - Students receive several oppor- 
tunities to pass the competency tests. An early warning system 
detects students who need special help. Students who fail the 
test receive remedial instruction. 

5. Adeguate Phase- in Time - Students receive sufficient warning (perhaps 
several years) before they have to demonstrate proficiency on com- 
petency tests. 

6. Sensible Setting of Standards - A systematic and, preferably, 
research-derived effort to set minimally acceptable scores for 
passing the tests is established. The standard-setting process 

is methodical and open to all concerned constituencies, especially 
minorities. 

7. Coordinated Staff Development - When the program is established, 
staff development activities help teachers and administrators to 
focus on strategies for student mastery of the targeted competencies. 

* 

It can be argued that the focus of most secondary schools is to teach 
students "how" to do. a few tasks (e.g., read a newspaper, write a letter and 
add the columns on their IRS Form 1040). In addition, secondary school curri- 
cula are often designed to teach students "about" the plays of Shakespeare, 
the discoveries of Marie Curie and the music of Beethoven. Students learn 
about literature, physics and music, but not necessarily "how" to write plays, 
study molecular functions or compose symphonies. 

In vocational-technical schools, however, the overall goal is not only 
to teach students math, reading and writing, but to help them apply that 
knowledge and acquire specific functional skills which will enable them to 
perform on the job. For this reason, it is also appropriate to devote some 
attention to the competency measurement efforts of the vocational education 
system. In some respects, the vocational education system is more like the 
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employment and training system than the "mainstream" secondary education 
system • 

B. The Vocational Education System 

Vocational educators have been benchmarking vocational skills for years 
although specific practices vary among localities. Many local programs have 
elaborate systems of advisory councils for the vocational areas in which 
they teach- The function of these councils varies, but the active ones 
often review and approve course curricula and even staff qualifications 
and equipment. Some states have tightly structured vocational education 
programs with statewide standards. Many programs integrate instruction in 
basic skills (e.g., science, math, reading comprehension) with vocational 
instruction. 

As Appendix 1 shows, vocational education makes up a significant part 
of the American educational system— the skills imparted at voc-tech schools 
are more closely and directly related to relatively clearly defined pro- 
fessions, jobs and employer needs than those at most general secondary 
schools. Writing in the American Vocational Journal , Ben Hirst, Jr. defined 
competency-based vocational education as "a systematic approach to instruction 
aimed ar accountability, based on job-derived standards, and supported by a 
feedback mechanism."^ 

Our research uncovered several descriptions of ideal processes which 
should be used to develop competency measures and benchmarks in the vocational 
education system. These processes and procedures can serve as model s for We" 
employment and training system in its efforts to develop benchmarks and com- 
petency measures. 

Although two of these systems are described in Appendix 1 , it is approp- 
riate to consider their common and essential elements here: 

io» 18 
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1. The basis for selecting the jobs for which training is to 
be provided and benchmarks developed is the demand .for 
workers. Training should only be provided in occupations 
which indicate high demand for workers. 

2. Jobs selected for curriculum development are analyzed to 
reveal the skills required, tasks performed and tools 
used. Initial research efforts should \?e validated by 
interviews with people currently filling the jobs. 

3. All competency measures and tests are validated and care- 
fully monitored during their initial implementation. 

*^ 

4. Competency measures and testing procedures are implemented 
gradually and with input from the relevant constituency 
groups. Adequate time is allowed for staff training be- 
fore final implementation. 

5. Regular efforts are made to review the competency 
measures and tests used. If appropriate, new measures, 
testing procedures or training curricula are developed. 

The development of competency-based programs was a long-term project 
of vocational -technical education, and there are a number of institutional 
manifestations of this relatively long-range interest in competency-based 
vocational education. Among these are the National Center for Vocational 
Education at Ohio State University and V-TECS (Vocational -Technical Educa- 
tion Consortium of States), a consortium of 17 states and several branches 
of the military. The National Center serves as a dissemination center "for 
collecting and exchanging curriculum materials and practices for vocational 
educators. V-TECS has produced catalogs of performance objectives (or 
competency indicators) and performance guides on specific jobs for use in 
the development of vocational -technical curricula. Appendix II lists the 
jobs studied by V-TECS and provides examples of task-specific performance 
guides.^ 



C The Development of Competency Standards and Benchmarks in the CETA System 

Implementation of most of the early programs funded by the Comprehen-^ 
sive Employment and Training Act was a somewhat hurried undertaking which 
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in most cases was not characterized by adequate long-range planning. The 
Initial concerns of early program operators were: (1). to establish appro- 
priate job training and job development strategies, (2) to implement opera- 
tional systems to support these activities, and (3) to insure some measure 
of accountability from service contractors. 

In a sense, the CETA system developed much like the education system, 
I'n early stages it provided basic services and training, and in later years 
more comprehensive and imaginative services were developed. As the employ- 
ment and training system matured, early concerns with "start up" of programs 
were replaced by a desire to increase the' efficiency and effectiveness^ of 
the programs funded under the Act, In later years, this trend was stimu- 
lated by criticism from those in the system and from outside the system. 

The critics of the early CETA system voiced many concerns, including 
those regarding the cost of providing the employment and training services 
and the ability of the system to produce graduates with useful skills. In 
an attempt to more efficiently and effectively provide needed services, 
program operators and program administrators adapted several management 
techniques not previously used in the CETA system. One of the more inter- 
esting techniques introduced was benchmarking. 

Benchmarking in- CETA youth programs has generally been conducted 
through various demonstrations. These programs include the Consolidated 
Youth Employment Program (CYEP); the Vocational Exploration Demonstration 
Project (VEDP); the Career Passport Project; the Skill Center Advanced 
Training and Employment Program (SCATE); and the Hartford Private Sector 
ImpToyn^^^ 

and to a lesser extent VEDP, were particularly helpful in the development 
of the OYP benchmarking paper. The following sections offer a summary of 
those two demonstrations, ^ 
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1. rh« Vocational Exploration Demonstration Project (VEDP) 

VEDP was a discretionary funded youth program designed to accomplish 
several goals: (1) to examine in a variety of program- models the relation- 
ships among program participants, program, activities and services, impacts, and 
environmental factors; (2) to compare summer and non-summer vocational explo- 
ration efforts; (3) to compare a vocational exploration experience over a 
twelve-month period with similar activities and services offered for shorter 
periods of time; and (4) to investigate the effects of vocational exploration 
programs upon the attitudes and institutional behavior of business and organized 
labor* 

The program was implemented in the summer of 1979 and ran through the 
1979-80 academic year. Summer, 1980, extensions operated in three of sixteen 
sites. Thirteen sites ran a separate summer, 1980, component; five of these 
also had a control group. This special demonstration project analyzed various 
vocational exploration program designs to determine what worked for whom, when, 
where, why and how. In a series of evaluation reports prepared by the St. Louis 
University Center for Urban Programs, the purpose of VEDP v/as described as one 
of assessing participants in the development of cognitive skills such as 
increased knowledge about the world of work and its range of careers; affective 
skills , such as better understanding of and adherence to the generally prevalent 
code of job behavior, and improved self-concept and interpersonal relations 
skills; and transition skills , such as riov.ing from the program to an unsub- 
sidized job or otheY. opportunity that helps in the participant's employability 

9 

development. Adjunct services are also provided, such as counseling, supportive 

services, educational enrichment, job development and assistance in securing 

post-program opportunities, and cdpJrTgTkilTsT - — — - ^ 

\ 

X 
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Vocational exploration was not seen as an end in itself, but rather 
as a "stepping stone" activity. Participation in a vocational exploration 
program involved the enrollee in a transitional process of growth and decision- 
making which was directed toward fostering in participants the appropriate 
attitudes, understanding, and appreciation of what is needed to successfully 
move from school to work and compete in the job market. 

Benchmarks for this program were established by a national task force. 
These benchmarks were related to one of four areas: (1) knowledge about kinds 
of work; (2) knowledge about the culture of work; (3) knowledge about and the 
skills for finding a job; and (4) knowledge and skills tor surviving in the 
work world. Youth enrolled in vocational exploration activities, and upon 
completion of the activites were tested for achievement of the benchmarks. 
These benchmarks were selected on the basis of several criteria: 

1. Each benchmark had to be directly related to at least one of 
the mandated objectives of vocational exploration; 

2. Each benchmark had to be" expressed in behavioral terms to en- 
hance the objectivity of the assessment system and to maintain 
the direct linkage of the benchmark to the program objectives; 
and 

3. Benchmarks had to emphasize behavior that demonstratas knowledge 
application rather than simply knowledge acquisition. 

2. The Conso/idatec/ youf/i Emp/oymenf Progrom (CyEP) 

CYEP demonstration projects operated during FY '80 in eight sites and in 
FY '81 at an additional five sites. CYEP attempted to test the feasibility of 
consolidating categorical youth programs (SYEP, YETP and YCCIP) into one, and 
providing truly individualized assessment and service delivery to youth partici- 
pants. Demonstration projects were designed to create a more flexible mechanism 
for delivering youth services to eligible participants. In designing the 
.Consolidated Youth Employment Programs, the Department of Labor required CYEP 
prime sponsors to create benchmarking systems which incorporated employer- 
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validated performance standards (benchmarks) for assessing individual par- 
ticipant achievements^. Those benchmarking efforts were intended to enhance the 
credibility of CETA participant achievements with local employers. The CYEP . 
benchmarking systems design was based on five major premises:^^ 

1. Special intervention is needed to facilitate the successful 

transition of low-income youth to the adult labor market 
especially youth who are minority group members, single 
parents or school dropouts • 

2. The needs of each eligible youth must be individually 
assessed to determine present interests, aptitudes, 
abilities and personal circumstances • 

3. An employability development service intervention strategy 
must be developed for each youth based on assessment 
results and the range of service options available, 

4- Gains in participants' abilities must be periodically 
measured and documented Using a set of locally estab- 
lished benchmarks for employability development which 
are relevant to the needs of the private sector, 

5. The program should be managed, based on specific levels 
of program performance to be attained, the degree to 
which youth are achieving individual gains, and by 
means of systems providing the information required to 
assess participant performance. 

a. The Assessment Process and EPR 

CYEP was intended to improve the effectiveness of employment and training 
services to youth by focusing on the assessment of the employability development 
needs of each participant and providing an individualized service intervention 
strategy documented in each youth's Employability Plan and Record (EPR)- In' 
the early CYEP days, most prime sponsors maintained an EPR and a Service Agree- 
ment for each par-^ticipant. The EPR was meant to be a long-term career develop- 
ment plan which reflected the aptitudes, talents and skills of each participant. 
The Service Agreement was intended to be maintained as formal documentation of 
the specific services to be provided to a participant and an agreement with a 
particular service agency to deliver those services. Designation of appropriate 



services to be received by a client depended on the employability goals and 
benchmarks identified in the EPR. 

According to the series of Quarterly Reports prepared by the Technical 
Assistance and Training Corporation (TATC), the early efforts at implementing 
CYEP benchmarking systems were characterized by several deficiencies; 

1. Most prime sponsor staff felt that the EPR meant more 
paperwork with no corresponding positive benefits to 

/ > the program or to participants. 

2. At some CYEP sites, prime- sponsors had the responsibility 
for preparing and maintaining EPRs. At some sites, both 
functions were delegated^to subrecipients. « To the extent 
that complete files were not maintained in a single loca- 
tion, using the EPR as a tool for planning was a cumber- 
some process. 

'3- At most sites, it appeared that EPRs, like the Employ- 
ability Development Plans (EDPs) which they replaced, 
encouraged an approach to employability development 
in which: 

a. Only one unit of service at a time was planned? 

b. Planning was restricted to what is provided by CYEP; 

c. Assessment, goals an?i activities were not explicitly 
linked; 

d. Employability development was onlV loosely tied to per- 
formance, standards, certificati,on .or credentialling. 

TATC found that EPRs were used to identify a participant's long-range 
goals, but few prime sponsors ventured beyond that point to explicitly define 
short-term objectives, performance benchmarks, etc. Referral to a unit of 
service toe often reflected the availability of a '*slot*' rather than a care- 
ful matching of the participant's present level of competency and interest. 
In theory, the EPR was intended to allow prime sponsor staff the opportunity to 
track client progress, necessitating frequent or at least regularly scheduled 
meetings with the individual client in order "to review progress and refine 
service rteeds. 
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Some improvement in using EPRs and benchmarking concepts was noted in 
later quarterly reports. However, the fourth quarterly report prepared by TATC 
noted that in a sample of participant records drawn from all of the CYEP sites, 
a planned service strategy was documented in only 7.1 percent of the EPRs 
sampled. At the time of the sample, only 18-7 percent (74) of those sampled 
had completed their initial activity, 13,4^peixeht^53) did not complete and 
4.8 percent (19) were no longer enrolled. 

Among the EPRs with sufficient data to conipare participant goals and units 
of service provided, the goals and' the units of service provided were consistent 
in 29.9 percent of "the cases. However, TATC observed that it w3is not always 
clear whether the units of service planned were a reflection of the youth's 
goals or whether the goals recorded were a reflection of the unit of service 
planned. At some sites, consistency appears to be a function of standardization 
and is accompanied by a loss of individualized planning in which the youth's 
own interests play a major role.^^ 

TATC*s evaluation of CYEP through the first quarter of FY '81 noted that 
the individualized planning and comprehensive assessment of client services was 
most easily accomplished by a centralized system of intake and assessment. 
Those prime sponsors who operated a centralized system were able to maintain 
adequate control over the quality and efficiency of the assessment process, 
resulting In more accurate indications of client needs and matching of appro- 
priate services to meet those needs. For most prime sponsors, however, assesa- 
ment appears to have been concentrated at client entry with only sporadic use ^ 
of this information to make decisions about what services a client should redeive. 
b. Developing the Benchmarking System 

the CYEP benchmarking process involved (1) developing competency benchmarks 
and benchmarking procedures; (2) measuring individual gains in competency through- 
out a youth's participation; and (3) documenting achievepients and awarding 
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credentials. Benchmarks were intended to reflect measurable competency 
Jevels* (colTipetency indicators) in each of four identified areas of employ- 
ability development; i*e,, educational skills,, pre-employment, work 
maturity and occupational skills, that are generally prerequisites f^r 
successful entry to the local labor market. Benchmarks were to be estab- 
lished at a number of proficiency levels to reflect the key steps to the 
attainment of entry-hiring r^uirements of local employers. 

CYEP projects experienced several problems in this development process 
One particular difficulty involved the use of planning task forces to - 
develop a benchmarking system. Task force members '.tend^ed to view .this 
task 'as secondary to tlieir regular responsibiVities. ^ As a result, task 
force meetings were difficult to schedule and ipembers rarely had assign- 
ments to complete between meetings, TATC noted 'that assigning benchmark 
'developmenj: to a single staff person with clear responsibility for their 
development within a . specified time period might have been a more efficient 
approach. Another groblem involved the initial conceptualization .of four 
benchmarking competency areas. ^ Pre-employment and work maturity descrip- 
tions contained common elements'; ..Beyond basic educational competencies,, 
educational attainment was also closely related to occupational or occupa- 
tional cluster skill requirements. Because of this confusion, none of the 
original eight demonstration sites V/ere able to desigrt benchmarks for more 
than two of the four competency areas. At all but five'sites, individual 
benchmarks were ill-defined or ambiguous. * Many benchmarks were vague and 
Included simple, and unelaborated completion measures such as "complete a 
specific unit of service, obtain relevant diploma or achieve satisfactory, 
attendance levels." 

Loosely defined and inconsistent benchmarks within a competency area ^ 
made it difficult for sitfes to assess participant strengths and weaknesses 
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in relation to benchmarks. In some cases checklists or other formats were 
used by counselors which identified acceptable standards in individual 
competency areas, but the scores to be achieved in order to move on to 
another area of competency were sometimes too ambitious and participants 
ended up ]?eing channeled into a predictable and similar sequence of ser- 
vice* , ^t^ 

Measuring the gains in competency acquisition was an informal process 
due in large part to the lack of clearly defined benchmarks. Because they 
were unable to pinpoint measures for benchmarks, sites^ad difficulty- in . 
developing methods to measure participant gains. The lack of well-defined 
competency indicators also constrained the developmental process both in 
terms of providing a particularly full range of assessment .services, and 
in reassessment of individual progress in skill acquisi tion. Local defini- 
tions for each of che four competency areas were established at all but 
two of the CYEP demonstration sites. Soma sites were able to break down 
and define the elements of these competency areas; unfortunately, only two 
sites were able to set benchmarks. The relationship of units of service 
to each competency area was also unclear. Several sponsors established a 
single benchmark in each competency area with no interim levels, other 
than completion of units of service to indicate progress of ^outh towards 
employability, 

D, Perceptions of and Support jar Benchmarking 

According to the Syracuse Research study of benchmarking practitioners; 
there are three general categories of reaction to the benchmarking concept: 
those who disagree with the concept; those who think the concept is good 
but that is is unworkable under current conditions; and those (the majority) 
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who support the concept and feel it is workable in the current employment 
, ancl training system. The support for benchmarking is even more encourag- 
ing because program operators seem to be saying "yes, this concept is 
worth developing despite the initial problems we may have had with im- 
plementing ^benchmarking systems.'* 

Private sector representatives contacted in the Syracuse Research 
survey were very supportive of the benchmarking concept and indicated 
particular support for the use of employer <:onsultation and validation of 
benchmarks. Other employers noted that benchmarking was a way for CETA 
prime sponsors to gain credibility in the private sector. Interviews 
with educators also resulted in expressions of support. Many argued that 
prime sponsors siiould^turn to educators for guidance in developing bench- 
marks since the educational system has had quite a bit of experience in 
developing competency-based training and performance benchmarks. 
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IIL Issues in the Development of a Benchmarking System 

In the fall of 1981 » the Syracuse Research Corporation (SRC) com- 
pleted a review of the state-^of*the-art of benchmarking in the CETA system. 
Their findings were based on visits to sponsors who had engaged in bench- 
marking efforts and regional seminars in each DOL region. In their three- 
volume report entitled Benchmarking: A State-of-the-Art Review 3 SRC 
identified the following as the four general purposes of benchmarking 
most often cited by CETA prime sponsors who were involved in developing 
and operating benchmarking systems: (1) to link the CETA system to 
employers by involving them in developing and validating benchmarks that 
communicates to potential employers the achievements and experiences of parti- 
cipants ir\ training programs; (2) to assist prime sponsors in managing their 
programs by providing detailed, and timely documentation of the operation of 
their service delivery system; (3) to facilitate program evaluations by 
determining goals, objectives and performance standards on which to assess 
program effectiveness; and (4) to assist client motivation by spelling out 

12 

the expectations for the individual participant within a training program. 

A well-designed benchmarking system should have the capacity to meet 
all of these prime sponsor needs, although current benchmarking operations 
are most likely to focus on only one of these aspects of benchmarking. This 
situation may be due to several factors: (1) limited organizational re^.ources 
(cuts in federal spending will undoubtedly affect the capacity of staff and 
other resources to implement and manage a benchmarking system), (2) lack of 
clearly-defined competencies and benchmarks, (3) insufficient coordination 
of services available through the deVvery system, and (4) inadequate 
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procedur^es and mechanisms for communicating information on participant 
progress and program operations. Often, these problems can be dealt with 
over time, as the benchmarking process is evaluated and refined. But 
several years of experience from implementing VEDP and CYEP benchmarking 
systems suggest some reasonable design parameters for the successful intro- 
duction of benchmarking into youth program operations. 

The benchmarking system, as we mentioned earlier, is comprised of four 
functional components: competency areas, competency indfcators, benchmarks, 
and assessment. While every benchmarking system is developed around these 
basic elements, significant differences occur in the focus of competency 
areas, the precision with which indicators are determined, the selection of 
benchmarks, and the design of procedures and instruments with which to assess 
performance. The most obvious differences are determined by the structure 
of the particular system in which benchmarking is applied, i.e., education, 
vocational education, employment and training. 

A. Operational Funi^ns of a Benchmarking System 

Three recent studies of benchmarking in the vocational education and 
CETA systems offer peel lent data about some of the successful examples of 

benchmarking designs. In essence, these designs describe benchmarking in 

I 

terms of six operational functions: 

1. Initial Assessment of client interests, aptitudes and short 
and long-t6rm goals (benchmarks) in addition to documentation 
of previous education and work experience; 

2. Employabil jtv Development Planning (EDP) in which assessment 
data and client-counselor interviews determine the types of 
services arid tv^aining that a client should receive in order 
to achieve agreed-upon goals; 
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3. Service Delivery based on service needs specified in the EDP, 
i.e. , remedial ed. , classroom training, work experience, OJT, 
vocational exploration, etc. 

4. Reassessment (testing and measurement) to determine whether a 
client has successfully completed an activity and achieved 
previously agreed-upon benchmarks, to review EDP and determine 
next steps in the employment process; 

5. Documentation which is based on a formal and coordinated system 
to collect, analyze and report accurate data on .individual client 
progress and general program- operations in a timely manner; 

6. Evaluation of client and program data to determine whether systems 
are operating efficiently and, in the long-run, to assess the 
effects of specific program strategies on the post-program 
experiences of former participants. 

Each of these functions within the benchmarking system implies specific 
needs in terms of staff capability and/or training; the structure of the 
prime sponsor organization in terms of planning, operations and administrative 
functions (including MIS and fiscal management); formal communication and co- 
ordination systems. The introduction of a new concept into already existing 
systems will generally be influenced by these organizational elements. The 
process of implementing CETA benchmarking systems for CETA youth programs 
will be discussed in Section IV of this paper. 

R identifying Competency Areas 

A competency-based employability development system must start with a 
broad consensus of what it takes to get and hold a job. Various projects 
have been funded to demonstrate the effects of benchmarking on promoting 
employability development services to youth. In FY*80 the Office of Youth 
Programs implemente^d the Consolidated Youth 'Employment Program (CYEP) to 
improve the effectiveness of youth services by combining all categorical 
youth programs into one and allowing clients access to activities as needed. 
CYEP focused on an individualized service intervention strategy that 
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identified client needs in terms of competency areas and assessed the 
achievement of specific competency indicators through the accomplishment 
of performance benchmarks. 

The results of projects like these have played an important. role in 
providing OYP with data that can guide them in decisions about the future 
potential of benchmarking and competency-building in measuring youth 
achievement. Although some difficulty was experienced by CYEP sponsors 
in their initial attempts to develop benchmarking systems, our review of 
the CYEP process evaluation reports revealed considerable support for 
using the competency areas identified there: (1) work maturity, (2) pre- 
employment, (3) occupational, and (4) educational. The selection of 
appropriate competency areas for youth employability development should 
be guided by several criteria: 

.1. Do the competency areas include all of the gene^^al 
behaviors that youth are expected to demonstrate? 

2. Is the set of competency areas manageable? Can youth 
attain competency in these areas? Are employment and 
training services available to assist youth in develop- 

• ing competency in these areas? 

3. Are the general desired behaviors de^scribed clearly for 
each competency area? Will people understand what is 
meant by the competency area? Will there be consistent 
understanding? 

4. Is the definition of the competency area free of word- 
. ing that suggests that some social, occupational or 

life roles should be valued more than others? 

In their quarterly evaluations of the CYEP demonstration projects, 
the Technical Assistance and Training Corporation described in detail 
the at1:empts of these projects to define the four competency areas. Few 
wesre successful at developing clear statements of purpose for each area, 
but several projects- were able to develop good general statements. Based 
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on a review and synthesis of the definitions developed by each of the 

CYEP projects, ALNA has assembled the following descriptions of four 

competency areas: 

1. Work Maturity - a demonstrated understanding of and ability 
to respond to the basic requirements of the work environment 
including attendance, ability to follow instructions, ability 
to work well with others. Work maturity should be measured 
during participation in classroom or WQrk experience-type 
activities through a participant's actual demonstration of 
levels of competency and documentation of achievements pro- 
vided by attendance records and written evaluations from 
work supervisors, trainers, instructors, etc. 

2- Edu cation - a demonstrated proficiency in basic academic 
skills sufficient to meet normally accepted achievement 
levels for a selected occupational area; a mastery of basic 
reading comprehension, writing and computations skills neces- 
sary for gaining and holding a job. A participant's minimal 
achievement of this competency should result in the attain- 
ment of a high school diploma or 6ED and ability to apply 
basic math and reading skills wichin one grade level of 
those required to perform a specific occupation as, defined 
by DOT. 

3. Skill Training - a demonstrated proficiency in knowledge and 
skills normally required to carry out entry-level tasks of 
the chosen job or mastery of basic (and advanced) skills re- 
quired for a specific occupation or cluster of occupations. 
Competency levels should be incorporated into the curriculum 
design, with measurement detennlned, by .attain of -techni- 
cal^ sktTTsaFd" credentials, where appropriate. These know- 
ledges and skills may be identified through a combination of 
DOT indicated and employer-validated competencies, 

4. Pre-Emp lo yment - skills and knowledge to identify career ob- 
ject ivesTur^^stand employer expectations and conform to 
job performance requirements so that an individual can seek 
and obtain employment. Pre-employment activities can include 
classroom sessions, seminars and tests in occupational goal- 
setting, interviewing, completing a job application, under- 
standing the local labor, market. 

While these definitions give one a sense of the kind of activities' 

involved in a competency area, they allow enough flexibility for local 

prime sponsors to determine which competency areas can be applied, based 

on local needs; select and adjust definitions of the relevant competency 
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areas so that individual competency indicators and benchmarks will more 
accurately reflect local employment and training activity. 

C. Assuming Ovmership at the Local Level 

Competency indicators should be selected and established to reflect 
local needs and the capacity of prime sponsor staff and resources to address 
those needs. While the National Office of DOL may define general behaviors 
that a CETA youth should demonstrate during the employability development 
process (competency areas), local prime sponsors must refine these definitions 
and determine what specific kinds of behavior are relevant to local hiring 
requirements and participant abilities. 

Before actual work in setting competencies can begin, there are several 
planning decisions that must be made by the prime sponsor director or other 
key decision-maker, the major decision being "what level of commitnent— 
in time, resources and money— am I willing to spend on this project?" 
This issue of commitment or ownership for the design, implementation and 
management of a new system is perhaps the most difficult hurdle to overcome. 
Even with federal mandates to support benchmarking, an adequate and useful 
benchmarking system will not be developed at the local level if the prime 
sponsor decisionmakers just "go through the motions." If local staffs are 
not made aware of the programmatic benefits which can be realized through 
the implementation of a benchmarking system, they can hardly be expected to 
enthusiastically support its implementation. 

The planning process should be developed by specific agency staff who 
are assigned the responsibility as part of their job rather than "in 
addition to" their other work. As a planning function, this responsibility 
might reasonably fall to a member of the planning staff. The plan should 
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be. developed with the cooperation of representatives of interested groups, 
including those providing service, those receiving service, employers, and 
the conmunity at large. In addition, a very crucial coordination issue 
must be addressed for insuring adequate communication and cooperation be- 
tween units in the prime sponsor^s own agency, particularly in developing 
a strong communciation and working relationship between planning, MIS and 
operations staff who all play important roles in the implementation'and 
management of the prime sponsor's benchmarking system. 

The question of which competency areas and competency indicators 
should be stressed can be a difficult one for some prime sponsors. Under- 
lying this question are two more basic questions: (1) what are the 
specific goals and objectives of the program, and (2) what is the service 
delivery capacity of the system? The development of specific and realistic 
goals and objectives for the prime sponsor is critical to determining the 
^ kinds of competencies that a CETA participant can be expected to demon- 
strate. Goals and objectives for a CETA delivery system are essential for 
establishing the parameters within which a prime sponsor will expect its 
program operators to perform. By setting these parameters, the. prime 
sponsor can better assess whether the services provided by its programs 
are i|ieeting the needs of CETA participants. Without some clearly defined 
goals to describe the prime sponsor delivery system and general client 
needs, it is difficult to develop appropriate competency indicators and 
benchmarks. 

D. Selecting and Setting Competencies 

Competency indicators should be developed on the basis of several 
criteria: 
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1, Does each set of competency indicators include all of the 
specific behaviors that youth must demonstrate in order 
to demonstrate achievement of the competency? Is the set 
comprehensive? 

2, Can youth attain the competency indicators? 

3, Does each competency indicator represent an important skill, 
knowledge or -attitude? Should each be included? 

4, Is each competency indicator expressed clearly? Will people 
understand what is meant? Will the understanding be con- 
sistent? 

5, Is the competency indicator applicable across lines of sex, 
^^socio-economic status, race, rural and urban settings, and 

reUgious persuasions in your community? 



The prime sponsor's plan for developing competency indicators and 
benchmarks should include the following categories of information: 
(1) objectives of the effort and expected products, (2) procedures to be 
used in completing the work, (3) responsibilities and authority of those 
who are to participate, (4) resources required to cojnplete the effort, 
(5) schedule of activities including completion dates, and (6) descrip- 
tion of how the results will be used. The plan should fit local condi- 
tions and reflect procedures that will meet local prime sponsor objectives 
within available resources. In preplaring a plan, consideration should be 
given to the following general processes. ^^'^ 

1. Informing people about the effort t this is a crucial step 
as people who know abput t^he effort are likely to partici- 
pate in it. This step includes preparing general informa- 
tion about the effort and getting the information to those 
who you want to be involved '^n the work. Particularly im- 
portant groups to involve in the information-sharing process 
are local planning councils (including the PIC an^ youth 
council), prime sponsor staff and prime sponsor contractors, 
subrecipi'ent^s^ etc* In addition to these more or less in- 
ternal groups, it is particularly important to make presenta- 
tions to local employers, since they should be involved in 
validating employability competencies. Perhaps more than 
any other group, it is vitaVto obtain the support of local ^ 
employers for the benchmarking system. 
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2. Collecting' information - this includes preliminary work of 
collecting ^descriptive information and instruments, carry- 
ing out data collection act^lvities and organizing and 
analyzing the data coll ectefd* The process of identifying 
appropriate competency indicators and benchmarks for any 
local priin^ sponsor requires that the prime sponsor develop 
and maintain contacts with local employers to learn their 
expectations of job applicants. By maintaining these con- 
tacts, itjmay be possible to virtually insure local employer 
support for the benchmarking system/ Prime sponsor planning 
staff must also design formal^ strategies for securing accu- 
rate data on local labor market demands and participant 
training and service needs* 

3. Drafting ithe initial set of competency indicators - in this 
step, tne data is transformed, into a simple, easy-to-read 
set of competency indicators. Criteria for determining 
appropriate competencies must be set, competency state- 
ments should be written and rewritten until they are easily 
understood by participants, the staff and employers. 

4. Validating the competency indicators - this step involves a 
review of the initial set of competency indicators by con- 
stituency groups (Especially local employers) to make sure 
that these indicators are the important ones and that the 
statements are understood. It isjalso possible to use this 
opportunity to have reviewers pribritize competency indica- 
tors. / ^ 

5. Preparing the final set - the final set of competency 
indicators is written, using information from the valida- 
tion process as the basis. , 

6. Communicating competency -indicators widely - perhaps the 
most important step in the effort is making sure that 
interested individuals and groups learn about the results 
of the effort. This is an opportunity for the prime 
sponsor to communicate information about the processes 
used to establish competency indicators, the content of 
the competency set and the procedures for implementing 
the benchmarking system. The form that this communication 
will take will vai^y with each group. For groups not 
directly involved with program operation or service 
delivery, .this .information can be provided as part of a 
publicity campaign, with brochures or a briefing session 
(perhaps the PICs and local employers might be the target 
audiences here). For groups involved in delivering ser- 
vices (like contractors, subrecipients). Information would 
be generated more formally through a prime sponsor execu- 
tive memo, written procedures on assessment and documenta- 
tion, a half-day workshop to train appropriate service 
provider staff in policies and procedures related to 
benchmarking, and provision of technical assistance by 
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/ appropriate prime sponsor staff. Of course, the staff of 
/ the prime sponsor itself must also be trained in the 

- benchmarking concept, policies and procedures, 

7^ — Upda^ting^piodicaljy - competency indicators should be 
reviewed and updatecl'TTt^regular intervals to respond to 
local conditions and needs. The updating process will 
probably not involve all of the same activities that re- 
sulted in the original set, but enough should be done to 
assure good input from key groups. 

The processes described above are those that a local prime sponsor 
should follow in order to establish competency indicators that are 
relevant to local conditions, A good preliminary step is to identify 
and ^study competency sets developed by others. These sets of compe- 
tencies should provide a starting point by introducing ideas and pro- 
viding experience-based .information to help along the thinking of 
planning groups. Appendices III and IV offer examples of competency in- 
dicators for each of the four competency areas which were identified by 
various CETA prime sponsors operating CYEP and VEDP benchmarking systems. 

Clearly stated competency indicators let those receiving service, 
those providing service, employers and the comnunity at-large know 
exactly what is being delivered through the program, Well-develope]cl and 
communicated sets of competency indicators improve the process of 
several service agencies sharing responsibility for employability de- 
velopment of individuals. 

Each agericy may concentrate on a portion of the competency indica- 
tors with. full employability development resulting from the efforts of 
the combination of agencies. This is efficient, fosters cooperation and 
eliminates duplication, A good set of competency indicators makes it 
possible to set and assess performance goals for an employability develop- 
ment system and to communicate the results back to interested individuals 
and groups. Accountability can be achieved in this way. 
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E. Diiermining Appropriate Benchmarks 

Benchmarking refers to a process of (1) setting employer-validated 
standards of employability for CETA participants; (2) assessing partici/ 
pant needs and interests with respect to those standards; (3) planning 
a program of acnievable benchmarks with each participant; (4) developing 
those services that will enable the participant to achieve his/her bench- 
marks; (5) systematically assessing and documenting participant progress 
toward and achievement of those benchmarks; and (6) enabling the partici- 
pant to effectively conmunicate his/her achievements to prospective em- 
ployers. Thus, benchmarking systematically ties together elements of^ 
existing CETA programs and can become a unifying concept for addressing 
the/perceived needs of the local delivery system for (1) increased access 
of CETA participants to private employers; (2) improved matching of CETA 
program objectives with the realities of the job market aind participant 
needs; and (3) accountability for the effect of CETA programs on the em- 
plpyability of participants. 

In a general sense, the actual development of benchmarks should rely 
on some framework for defining, measuring and documenting the achievements 
of each individual participant. Once the prime sponsor has determined 
which competency areas are significant to the development of its partici- " 
pants and has selected competency indicators for each of these areas, it 
must design a standard approach to be used to measure and* document the 
attainment of these competency indicators. 

Consistency and standardization of benchmarks for each identified 
competency indicator is important in order to make benchmarking a useful 
tool for conmunicating a youth *s* experiences and achievements to employers, 
educators, and others in the community. Benchmarks also make it easier 
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for participants and program ?taff to communicate to each othev* what they 
believe to be reasonable and "doable" achievements for the participant. 
In some instances, benchmarks are numerical measures of work achievement, 
like reporting to work on time for nine out of ten days. In other areas, 
benchmarks are determined through observations pf behavior in a work or 
classroom setting or in some controlled situation like a role-play or mock 
interview. In still other areas, benchmarks are defined by how well a 
cTient scores on written tests of skill or aptitude. 

The identification of specific performance benchmarks should involve, 
a consultative process -similar to that used to identify and define compe- 
tency indicators. Decisions about which benchmarks to identify cannot be 
made in isolation. The selection of .benchmarks must consider local labor 
market conditions, employeV expectations, participant needs for employ- 
ability development and the capacity of the prime sponsor delivery system 
to provide appropriate employment and training services. Benchmarks 'must 
re/lect real outcomes so that their achievement indicates that a CETA par- 
ticipant is job-ready. In identifying appropriate bencjimarks, a prime 
sponsor should concentrate on three major decisions: 

!♦ What competency areas should be stressed or at least 
addressed first? 

2. Within a competency area, which specific competency 
indicators should be singled out for benchmarking? 

3. For every competency indicator selected, what behavior • * c 
indicates attainment of the competency? 

With the prime sponsor staff again playing the key role as coordinator and 

manager of the development process, benchmarks should be determined in the 
15 

following manner: 

I. Review each competency indicator and determine that 
each is defined in terms which are observable and 
measurable behaviors and attitudes. 
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Z* Translate the definitions into behavioral learning ob- 
jectives, i,e,» criteria and benchmarks, which represent 
the minimum skill levels for certification in the com- 
petency area. 

3. Refine objectives, drawing upon input from appropriate 
advisory groups to ensure that definitions and objectives 
reflect the standards and expectations of the local labor 
market and speak the language of the private employer, 

4* Identify or develop, appropriate. assessment techniques, 

5* ^Design or select appropriate assessment instruments, 

6* Incorporate appropriate assessment criteria into forms 
and pirocedures related to assessment and employability 
development plans (EDPs). ,^ | 

7. Develop appropriate certification and documentation systems, 
1. Reviewing Appropriate Competency Indicators ^ 

The acquisition of various competency indicators is defined by a 
participant's successful completion of specific educational, experiential 
and training activities which the participant, counselor and other appro- 
priate prime sponsor staff have agreed are important for that individual's 
successful transition to employment. The development of standards against 
which to measure participant success involves reviewing each competency 
indicator to identify the performance of some activity or the demonstra- 
tion of some attitude which signifies a certain level of proficiency. 

In general, competency indicators should be clearly defined state- 
ments that identify knowledge of content, function or performance and 

■« ifi 

personal characteristics. Knowledge competencies require some kind of 
information about a particular subject or topic (e,g,, knowledge of 
potential employment prospects in three local firms; knowledge of entry- 
level skills for a chosen occupation; knowledge of health and safety 
requirements on a job). Function competencies require the ability to 
perform some prescribed action (look through the classified ads and 

^ ■ 

f 

A. L NELLUM AND ASSOCIATES -33- i 



4 

Identify five possible job openings in the client's chosen occupational 
field; report to work on time; assemble and disassemble a circuit board; 
read a blueprint). Personal characteristics cut across both of these 
dimensions and- identify personal qualities or attributes such as intelli- 
gence, honesty and conscientiousness that are desirable in the perform- 
ance of any job- 

2 . Translating Definitions into Behavioral Learning Objectives . 

Once competency indicators have been reviewed and, if necessary, 
revised into clear statements of knowledge, performance or personal 
characteristics, these statements must be broken down into behavioral 
learning objectives. These objectives are generally developed through 
consultation with local employers to determine what they consider appro- 
priate job-related behavior, consultation with LEAs to determine minimum 
general education levels., and review of training curricula to identify 
criteria for measuring proficiency in specific occupational and educa- 
tional skill areas. 

For example, if employers say that reliable attendance is an important 
attribute looked for in potential employees, benchmarks would be estab- 
lished for documenting participants' attendance and determining a specific 
range of performance that could be measured, i.e., excellent,' good, fair, 
poor; or acceptable/unacceptable. Another example would be to identify 
behavioral objectives for a learned skill like job finding. The ability to 
find a suitable job in a reasonable period of time demands a series of 
learnable skills, including (1) how to access career and occupational in- 
formation, (2) how to write a resume and fill out an application fornti, and 
(3) how to respond in an interview situation. Learning objectives could 
be developed to assess the acquisition of each of these skills by requiring 
that the participant be able to (1) identify three occupations in a chosen 
field and describe the responsibilities and duties of each, (2) write a 
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resume with no spelling errors or fill out an application neatly and 
accurately, and (3)' participate in two mock interviews and demonstrate 
a thorough ability to effectively communicate and relay relevant infor- 
mation. 

It is important here that appropriate prime sponsor staff are able 
to provide the benchmarking work group with sufficient data to assist in 
this decision-making process. These data might include benchmarks that 
have already been developed by other prime sponsors, curriculum outlines 
or lesson plans that identify learning objectives of the appropriate 
skills training courses, skill requirements identified in appropriate 
apprenticeship programs, etc. 

3. Refining Objectives 

Refinement of the selected learning objectives involves reviews by a 
larger group of "advisors" like a youth planning council, PIC or other 
volunteer body of educators, employers, CETA program operators, etc- 
This process should again consider whether the benchmarks represent 
realistic and achievable outcomes of the prime sponsor's youth programs, 

4. Identifying Appropriate Assessment Techniques 

Benchmarks must be defined in terms which can be directly quantified 

18 

and measured or for which we can develop proxies to represent them. 
For example, if the prime sponsor identifies an occupational skills 
benchmark as "the ability to answer, with 70% accuracy, a test which in- 
cludes mathematical computations performed as a cable TV installation 
technician," the achievement of those benchmarks can be measured by the 
participant's score on the tesc. If another benchmark is "to carry out 
instructions in an e)«)editious manner," some proxy for measuring this 
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benchcnark must be developed, like carrying out the instructions and com- 
pleting the task in 15 minutes with no errors. 

The measurement of these benchmarks is performed through some method 
of assessment and the use of' appropriate assessment or testing instru- 
ments. Numerous assessment approaches are available, but when you ''boil 
it down," there are only a few essential differences among them. These 

differences revolve around the three basic components of any assessment: 

. , 19 
Stimulus, response and scoring method. In all assessment methods, some 

stimulus is given to the examinee (e.g., a paper with questions on it or 
parts of a machine needing to be assembled) which requires some response 
(e^g., writing the answers to questions or assembling the machine) which 
will be scored according to some established criteria (e.g., answers 
given are compared to a list of previously established correct answers or 
the machine's smooth and efficient operation is judged.) 

Various combinations of stimulus, response and scoring methods have 
evolved over time for measuring competence. Three basic types of assess- 
ment methods exist: 

\ 

1. Behavioral observati on is most frequently used to measure 
a participant's explication of a skill, knowledge dr 
attitude on the job or in a simulated situation. Examples 
of behavioral observation would be a work supervisor\s 
assessment of a participant's punctuality on the job or 

an assessment of an individual's ability to coimiunicatfe, 
during a mock job interview. 

2. Product reviews are used to assess whether a participant 
can demonstrate that he or she possesses a particular 
skill, e.g. 5 write a report, assemble a carburetor. 

3. Oral or written questions can assess whether a participant 
has acquired knowledge or can apply the knowledge acquired 
by solving a problem. 
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In determining which assessment method to select, the prime sponsor 

should consider (1) the nature of the competency indicator and benchmark 

(a skill, knowledge or attitude) and (2) whether the benchmark will assess 

a process people use or a product they produce. 

Designing or Selecting Appropriate Assessment Instruments 

It is worthwhile to consider using an existing assessment instrument 

rather than developing your own because it generally will be less expensive 

and certain aspects of the instrun^nt's technical quality are likely to be 

higher. Existing instruments are available from publishers or persons who 

have developed their own for a specific use but have never published them. 

Published tests are generally extensively reviewed, pilot tested and revised 

before publishing, although they may not have all the characteristics important 

20 

for the particular situation. Prime sponsor staff should review these testing 
materials to determine whether they meet the needs of the particular competency 
areas involved. 

Three general categories of criteria should be reviewed when selecting 
assessment instruments: 

1. Validity , or the extent to which the instrument measures 
the skills, knowledge or attitudes looked for. This is 
the most crucial issue in selecting an appropriate instru- 
ment particularly if you are considering using one that is 
already developed. Each question should be carefully re- 
viewed to determine whether a correct answer to that ques- 
tion will help to determine a participant's achievement of 
a benchmark, ^ 

2. Usability, or the extent to .which the assessment instrument 
is suitable for administration by current staff if it 
must be administered by a certified staff person. Does the 
test measure a skill level which can be attained by partici- 
pants? 

3. Reliability , or the extent to which an instrument is accurate 
in Its measurement of a level of knowledge, skill or an 
attitude. In general, objective (paper and pencil) tests 
are quite reliable and accurate as measures of particular 
knowledge. Tests of attitude are considerably less reliable. 
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6. Designing Forms and Procedures 

Once appropriate assessment criteria have been selected, forms and 
procedures must be designed to document assessment and benchmarking activities. 
Aside from the paper and pencil tests referred to in the previous subsection, 
there will be other internal reporting forms that need to be developed for 
counselor interviews, job supervisor evaluations of participant activity, 
other types of participant performance evaluations, or individual participant 
progress reports written by service deliverers after a youth has completed a 
particular unit of service. Procedures should be developed for completing the 
forms in a timely manner and ensuring that these documents are maintained in 
client folders. 

The achievement of specific benchmarks and attainment of competency indica- 
tors must be documented in the client's EDP. When attainment of a competency 
indicator is also translated into completion of a specific CETA program activity 
(i.e., work experience, ESL course, electronics technician training, etc.)»- 
that achievement is also documented on prime sponsor MIS forms, like a change 
of status form or a termination form (in the case of a participant's comple- 
tion of CETA). 

7. Developing Appropriate Documentation Systems 

Decisions must be made about what internal prime sponsor processes need 
to be established or refined to appropriately document participant achievements. 
These systems need to identify the types of information that need to be docu- 
mented; the forms, checklists or narrative reports in which this information 
should be reported; the frequency with which this information should be docu- 
mented (monthly, quarterly); and the individual (s) who should be responsible for 
documenting the information. 
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Prime sponsors are required by the CETA regulations to establish 
certain basic systems for docmnenting participant progress. In most 
cases, the Implementation of a benchmarking system will require some 
revisions in existing procedures to allow "^or the introduction of such 
benchmarking concepts as reassessment, employer-validation and more de- 
tailed EDP planning, 
8* Credential ling 

Credential ling, or certification, documents a youth's achievement 
according to a standard set of benchmarks which should enable that par- 
ticular youth to successfully enter and compete in the labor market. 
As a youth achieves benchmarks towards a competency, he or she is 
building a resume for the job search effort. 

If prime sponsors are addressing educational or occupational skill 
areas for which established credentials exist (i,e,, high school diploma 
or GED; apprenticeship certificate, etc), benchmarking designs should 
focus on competency indicators that will lead to those credentials. It 
is very important for prime sponsors to work with employers and creden- 
tialling institutions to have benchmarks accepted for credentialling pur- 
poses. In other competency areas where there are no existing credential- 
ling systems, prime sponsors should attempt to establish the validity of 
their benchmarking systems as reliable measures of achievement so that 
they too can be accepted and used by prospective employers, educational 
institutions, etc, that would require documentation of CETA participants' 
achievements. 

By developing benchmarks to satisfy the specific requirements of 
each set of competency indicators, identifying the methods and instruments 
to assess participants* achievements, and designing appropriate processes 
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for documenting these achievements, participant progress can be 
monitored and service needs can be revised to ensure participant 
chances for an adequate job placement. Appendix IV of this report 
offers some examples of benchmarks developed by the St, Louis Univer- 
sity Center for Urban Programs for a VEDP benchmark assessment manual 1 
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IV. Implementing a Benchmarking System: Selected Issues 

f 

The implementation of the benchmarking concept and systems design 



r*gquires decision-making at three levels: 

1. The national level , where the concept is defined, basic 
elements and procedures are Identified, and policy 
guidance is provided; 

2. The regional level , where technical* assistance and training 
are made available to local prime sponsors; and 

3. The local level which accepts this guidance, implements 

• procedures based on the capabilities of prime sponsor staff 
and resources, and monitors and evalujites^ prggram^^per-formance^ 
to insure that the b^enchmarking system is operating ef- 
ficiently. 

A national benchmarking requirement will have a greater chance for 
successful implementation if uniform systems criteria for local prime 
sponsor adoption are provided. Regulatory language and explicit guide- 
lines should address such issues as: (1) the definition of the concept; 
(2) the identification of benchmarking as a priority activity; (3) a gen- 
eral description of regional and local responsibilities for implementation; 
(4) the specific requirements for integrating or redesigning existing 
operational systems to accommodate benchmarking processes (i." , the 
development of formal procedures for assessment and reassessment of par- 
ticipant progress; the assessment or progress based on the achievement of 
specific benchmarks; the development of formal procedures for documenting 
progress in EDPs, etc.); (5) the involvement of advisory groups to develop 
local competency indicators and benchmarks; (6) the requirement for ob- 
taining employer-validated competency indicators and perhaps some options 
for how that information can be obtained; (7) the requirements for docu- 
meriting the benchmarking process; and perhaps (8) a discussion of a 

K 

reasonable implementation timeline. 
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An important issue to be considered in the implementation of a 

national benchmarking system is the level of specificity with which the 

DOL national office can reasonably expect to define an acceptable system 

and develop requisite monitoring and documentation standards. A review 

of relevant benchmarking literature— both within the employment and 

training system and outside it— offers some guidance for the Office of 

Youth Programs. Specifically, Benchmarking: A State-of-the-Art Review , 

by Syracuse Research Corporation, and Progress Toward Integrating 

Services for Youth , by TATC, provide some excellent observations. CETA 

program operators who have attempted to implement benchmarking systems 

have voiced common concerns regarding the relative roles of the regional 

21 

and national DOL in providing guidance. Interviews and workshops by 
the Syracuse Research Corporation indicate that prime sponsors want a 
clearer definition of (and perhaps closer involvement from) the roles of 
the national and regional offices, but they did not want them to "infringe" 
too much on local control. Prime sponsors want enough structure from the 
national and regional offices but they are also concerned that a nationally 
mandated benchmarking system might not be flexible enough to meet local 
priorities for ciient needs, staff resources and local delivery system 
capabilities. It is also essential for the Department of Labor to care- 
fully consider the costs which will be incurred in implementing a bench- 
marking requirement. If these costs are substantial, DOL must provide 
for this in the systems implementation plan. ^ 

Given these particular caveats, there are some specific criteria that 
should be considered in a benchmarking systems design: 
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L It should identify competency areas which define those 
general behaviors that a youth must develop in order 
to make a successful transition into the labor market; 

2. It should identify minimum competency indicators which 
represent specific skills, knowledge or attitudes for 
which benchmarks will be developed; 

3. It should require benchmarks which are locally-based 
and developed through consultation with employers, the 
community, labor organizations, local education agencies, 
and other relevant groups who can provide input into 
defining those skills, knowledges or attitudes 'which 
must be learned in order for an individual to obtain em- 
ployment; 

4. It should define benchmarks in terms that are quantifi- 
able and measurable; 

5. Itf should include measurement instruments for each bench- 
mark; 

6. It should provide methods and procedures for documenting 
a participant's accomplishment at meeting benchmarks and 
Acquiring competencies; 

7. It s|iould provide some credential or other documentation 
of a participant's achievement of competencies which the 
individual participant can present to an employer. 



The Syracuse and TATC reports represent a good first effort at identi- 
fying systemic problems that s?*^faced during the implementation of bench- 
marking in a selected number of CETA prime sponsors. Syracuse concluded 
from their interviews that problems with systems implementation were based 
on the lack of any system-wide definition of benchmarking. The lac^of 
adequate definition resulted in a variety of perceptions about what bench- 
marking was supposed to do. In addition, TATC quarterly reports of CYEP 
benchmarking revealed that, although prime sponsors were provided with a 
planning package and technical assistance from national or regional office 
staff, each prime sponsor was generally left on its own when it came to 
actually designing and implementing the benchmarking system. Any future 
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efforts toward implementfng a national benchmarking requirement should 
address these issues of clearly defining key concepts and terms, and 
providing adequate technical assistance, training and financial support 
to local -program operators ^ 

Syracuse Research analyzed six operational issues which concerned 
prime sponsors about the implementation of benchmarking procedures: 
timing, resources, scope, design, impact on other operational systems, 
and staff training and capacity. These issues are particularly important, 
given current budget restrictions, cuts in staff and other local resources, 
and the still unresolved future of CETA programs in general . A brief 

V 

discussion of each of these issues follows. v 
A. Timing 

tiearly all of the prime sponsors interviewed by Syracuse were con- 
cerned that there be enough time allowed for them to implement the system. 
A suggested time period was approximately two yeeirs, which would allow 
prime sponsors to phase in the new system. The first year would concen- 
trate on development of systems, policies and procedures. The second 
year would focus on implementation and systems refinement. The discussion 
of timing is, of course, Jinked to the complexity of the benchmarking 
system to be implemented. Simple designs would take less time and re- 
sources to implement than more sophisticated ones. This, in turn, is 
affected by a particular prime sponsor *s resources, size and scope qf 
the delivery system. With two years of CYEP benchmarking just completed, 
the findings from the TATC evaluation provide some useful information re- 
garding the issue of timing. 
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B» R€$ource9 

The amount of money that prime sponsors have to operate youth programs 
has an impact on the types of benchmarking practices that they utilize* 
Syracuse Research noted that most prime sponsors felt tha-t implementation 
of a new operational system would require financial support over and above 
what was allocated for normal program operations* Most prime sponsor^s felt 
that they should not be askec^ to **impl anient a benchmarking system unless 
adequate resources were provided. The phase-in approach (over a period 
of two^ years) was suggested as one way to spread developmental costs over 
time, thereby lessening the financial strain that would result from 
having to implement a system in two or three months, 

C Scope 

Prime sponsors need to decide whether all youth programs, including 
Titles II-B and VII, should be included in the initial implementation of 
benchmarking systems. Some prime sponsors argued that benchmarking should 
not be applied to SYEP programs; for example, since they are short-term 
activities to which long-range developmental measures could not be applied. 
Other prime sponsors believed that SYEP could benefit by the structure 
offered in benchmarking and that limited, achievable objectives for youth 
programs should be developed. 

D. Design 

The issue of program design focused on (1) the variations in the de- 
sign of intake and assessment procedures, (2) employabil ity development 
planning and recording, and (3) documentation and certification procedures, 
MIS procedures, etc. 

Very briefly, prime sponsors indicated a wide variety of systems 

designs and procedures for assessing client needs and assigning services. 

/ 
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For some prime sponsors, the assessment process Involved a minimal amount 4 
of testing and counseling, with the EDP used to document initial assess- 
ment activity but with little refinement beyond intake. Services in some 
prime sponsors were assigned sequentially (e,g,, ESL before a classroom 
training assignment), although there was no indication that reassessment 
was made upon the completion of one activity and assignment of another. 
As a result, benchmarking was used more in forni than in content; client 

; records .were documented in a "spotty" manner. Because benchmarking is 
focused on a long-range, developmental approach to the delivery of em- 
ployment and training services, prime sponsor directives should reinforce 
the importance of assessment and reassessment, accurate documentation in 
the client EDP, and closer scrutiny of what it will take to prepare a 
particular individual for employment. Prime sponsors should attempt to 
refine their assessment procedures and perhaps review operations in terms 

> of who should assess clients, what kinds of tools should be used (interest 
or aptitutie tests> work sampling, etc), what assessment techniques are 
appropriate for a particular sponsor client population, what types of in- 
formation should be recorded in {he EDP, how much should the client's 
development be documented in the EDP, etc. If the benchmarking system is 
to be used as a means of providing the CETA client with suitable employ- 
ment credentials, these issues need to be addressed. 

E. Impact on Other Operational Systems 

Benchmarking implies that a prime sponsor can develop and maintain 

I 

several management and administrative systems: 

1. an MIS that cjan collect benchmarking data and produce 
usable reports; 



ERIC 



a,;l nellum and associates 



■46- 



54 



2. a planning and evaluation capability that can determine 
what kinds of benchmarking data can be used tjo evaluate 
program performance i ! 

3. a service delivery/client flow system that is adjusted 
to accommodate various elements of the benchmarking 
process (i.e., a "tighter" assessment system that al- 
lows for scheduled reassessment of clients throughout 
their participation in CETA); 

4. clearly defined staff roles and organizational responsi- 
: bilitles that identify, for example, the relationship of 

operations staff to MIS and planning in terms of monitor- 
ing the performance of the benchmarking system. 

Because benchmarking is intended to influence all of these organiza- 
tional units, it is necessary that local prime sponsors are provided with 
sufficient time to prepare their existing systems for the introduction of 
benchmarking. A work group of key staff should be identified whose 
responsibility it would be to review the developmental tasks of imple- 
mentation and advise prime sponsor decision-makers on appropriate policy 
and procedures. 

St€xff Training and Capacity 

The successful introduction of a new management system into any 
organization mu^t be preceded by appropriate staff training and prepara- 
tion of companion management systems. 

Many of the prime sponsors interviewed by Syracuse Research Corpora- 
tion indicated a general lack of in-house expertise in establishing or 

22 

operating their benchmarking systems. Prime sponsors expressed a need 
for technical assistance and staff training, and indicated support for 
an "extended" period of planning and technical assistance. One sugges- 
tion discussed in the section on "timing" in this paper, proposes a two- 

i 

phajsed approach that could accommodate sufficient time for systems 
development. [See Section A] 
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For many of the prime sponsors interviewed by Syracuse, there was 
particular concern regarding whether cuts in local operating budgets 
and resulting cuts in staff would preclude anything short of the above- 
mentioned phase-in approach. By phasing-in staff training and systems 
development over an entire fiscal year, prime sponsors may be better 
able to introduce procedures, train staff in implementing those proce- 
dures and '*^,est" their operation as well as maintain existing service 
delivery activities at an adequate level- Staff training during the 
developmental phase could be provided by regional DOL staff to selected 
staff of prime sponsors throughout the region and perhaps followed-up 
by technical assistance to individual prime sponsors. 

Aside from providing adequate staff training on the design and imple 
mentation of a benchmarking system, prime sponsors need to review exist- 
ing organizational units and systems to determine what effects benchmark- 
ing will have on the;ir operation. Some of the salient features of a 
benchmarking system include (1) assessment and regularly scheduled re- 
assessment of participant progress, (2) frequent testing and measurement 
of achievement of benchmarks, (3) consistent documentation of achieve- 
ments in client EDPs., a^d (4) regular evaluation and employer validation 
of competency indicators^ and benchmarks. While most of these features 

already exist in some fohf\ in the prime sponsor system, they will, no 

\i 

doubt, need to be revised^ to accommodate procedures for developing, 

\ 

measuring and documenting! \Competency indicators and benchmarks. 

The introduction of benchmarking will also require revision of 
staff job descriptions so that responsibilities for the operation and 
maViagement of the benchms^rking system are already assigned. Divisions 
of labor and identification of tasks might result in the following types 
of staff assignment: ' 
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1. Planning staff to conduct labor market research, provide 
direction and staff support to benchmark work groups, 
conduct employer validation process, 

2. Assessment;! staff to provide regularly scheduled partici- 
pant interviews, assist participants in EDP planning, 
administer! and score assessment tests, document bench- 
mark achievements, select appropriate participant services, 

3- Training staff and worksite supervisors to provide feed- 
back and evaluations of participant performance/achieve- 
ment of benchmarks, assist in validating curricula to 
determine that it meets employer expectations, reflects 
Hocal labor market needs, and enhances participants' em- 
ployability development, 

4. MIS staff to determine whether the introduction of bench- 
marking may require some additional data collection re- 
quirements or revisions in definitions of participant 
status (e.g., perhaps an additional item on "change of 
status" form might be, "achieved benchmark" or "achieved 
competency in X " or "acquired credential''), 

5. Monitoring staff to review the performance of benchmarking 
at regular intervals, determine that all systems require- 
ments are functioning (e.g., properly documented EDPs, 
adequate testing/assessment facilities, regularly scheduled 
reassessments for participants). 
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V. Findings 



Based on our research, the following appear to be among the more 
prominent features which a nationally-mandated benchmarking system 
should possess: 

1. Regulations promulgated by the National Office of the 
Department of Labor which: 

a. define the concept; 

b. specify basic terminology, such as competency 
area, competency indicator, benchmarks, assess- 
ment; 

c» determine the scope of the effort, i.e., will 
benchmarking be mandated for all youth activities 
or only Title IV; 

d, identify roles and responsibilities; and 

e. indicate standard policies and procedures. 

2. A national information network which allows benchmarking 
practitioners the opportunity to share experiencies and 
expertise. 

3. Technical assistance and training which are provided by 
regional office staff to local prime sponsors. 

4. Responsibility for system implementation and operation 
which is delegated to local program operators. Local 
program operators should be allowed sufficient training, 
financial resources and time to develop and implement a 
benc)imarking system. 

5< Competency indicators and benchmarks which are: 

a. selected and validated with local employers; 

reviewed by local employment and training 
advisory bodies; and 

c. re-evaluated on a regular basis. 

6. Benchmarks which are quantifiable measures of profi- 
ciency in a given competency area. 
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7. Benchmarks which establish standard measures within a 
given competency indicator. When appropriate and possible, 
several benchmarks should be established for each competency 
indicator. 

8. Benchmarks and competency achievement which are tied to 
specific objective measures of attainment, 

9. Assessment tools and procedures which: 

a. identify participant needs; 

b. select appropriate services to be delivered; 
and 

c. monitor participant achievement of benchmarks, 

10. Assessment goals, activities and services which are clearly 
related to an individual's emplcyability development plan 
(EDP) . 

11. Employability development plans which document participants* 
short-term and long-term employability goals and achieve- 
ments. 

12. Local incentives (including sufficient financial support) 
which encourage prime sponsors to develop and maintain 
adequate benchmarking systems. 
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"Benchmarking" as discussed in this paper should not be confused with 
"performance benchmarks" used in FY 1982 grant review guidelines for 
CETA Title IIBC programs. IIBC performance benchmarks proposed a 
methodology allowing regions to calculate standards against which to 
review and analyze prime sponsor annual plans. The methodology took 
into account such local performance variables as participant charac- 
teristics, program mix, labor market and other local economic condi- 
tions. 

2 

Benchmarking: A State-of-the-Art Review, Vol. II , Syracuse Research 
Corporation, New York, May 1981, p, 13, 

3 

Ibid,, p. 13, 

See also : Progress Toward Integrating Services for Youth, Sixth 
Quarterly Report , Technical Assistance and Training Corporation, 
Washington, D.C., January 1981. 

This report of CYEP program performance during the first quarter of 
FY 1981 offers several observations about problems encountered during 
CYEP implementation and the development of "formal" benchmarking sys- 
tems. See specifically Sections II and III. 

4 

See: Eric Butler and James Darr, Research on Youth Employment and 
Imployabil ity Development: Educator and Employer Perspectives , Youth 
Knowledge Development Report 2.16, Washington, D,C., May 1980, p. 7-8. 

Employers appeared repeatedly to advise CETA prime sponsors that in 
order to develop employabil ity in an individual, CETA programs need 
to assist participants to: (1) gain work experience, (2) mature in 
work attitudes, (3) acquire basic educational competencies, and to a 
lesser extent (4) acquire occupational competencies. 

A 1977 survey of 500,000 small business member;" of the National 
Federation of Independent Businesses (NFIB) found that employers 
shared the following concerns about job applicants: (1) lack of job 
skills (39 percent of survey respondents), (2) won't stay long on the 
job (39 percent), (3) can't read (13 percent), (4. lacks good appear- 
ance (21 percent). Another NFIB survey conducted in January 1981 
reported that "finding qualified employees" ranked among the top ten 
problems confronting employers after interest rates, cost of insurance, 
cost of energy, cost of labor and local tax rates. 

^Paul Thurston and Ernest R. House, "The NIE Advisory Hearing on 
Minimum Competency Testing," Phi Delta Kappan , Vol. 63, No. 2, 
October 1981, p. 87. 
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W. James Popham, "The Case for Minimum Competency Testing," Phi Delta 
Kappan > Vol. 63, No. 2, October 1981, p. 90, 

"^Ben A, Hirst, Or,, "The Components of Competency-Based Vocational 
Education^" American Vocational Journal , Vol- 52, No. 8, November 1977, 
p- 32* 

See : Benchmarking: A State-of-the-Art Review , op. cit*. Vol. II, 
pp. 46-48. 

9 

' See: St. Louis University Center for Urban Programs, The Vocational 
Ixploration Demonstration Project: An Analysis of the 1979-80 In- 
School Extension Components , April, 1981 > 

^ ^CYEP GoaU Mt^ nt ioned in EDPs , For the CYEP sample as a whole, 29 per- 

cent of the EPRs did not document short-range goals. The remaining 

records contained 566 goals. Most frequently rated as short-term 
goals were: 

Gain work experience 15.7% 
Complete high school 15.2% 
Acquire good work habits 12.9% 
Define career goals 10.1% 
Obtain (specified) skill training 9.7% 
Obtain GEO 7.1% 
Obtain (unspecified) training 6.4% 
Learn job search skills 5.5% 

Approximately 44 pi^rcent of the goals mentioned were pre-employment 
related, 29 percent education related, and 16 percent were training 
related. At four sites, the highest percentage of the short-term goals 
were pre-employment related. At three sites the highest percentage of 
the goals were education related. 

In the CYEP sample, 32 percent of the EPRs did not record long-range 
goals; 12 percent recorded that the youth were undecided. In the re- 
maining records, 252 goals were recorded: 

Specific technical or paraprofessional occupational 18.3% ^ 

Specific unskilled or semi-skilled occupational 15.0% 

Specific professional occupational 12.3% 

Clerical occupational 12.3% 

Specific skilled occupational 10.3% 

Attend college 7.9% 

Attend other post-secondary school 4.4% 

Enter military 4.4% 

^^In their study, Syracuse Research Corporation conducted visits to selected 
prime sponsors in each of the ten federal regions. In all, their research 
involved contact with each of the CYEP sponsors, and with many interested 
individuals in each region. 

12 

Benchmarking: A State-of-the-Art Review , op. cit, pp. 16-17, 53-61. 

fSi A, L NELLUM AND ASSOCIATES -53- ^1 



See : a) Benchmarking: A State-of-the-Art Review, VoK I., II, III > 
Syracuse Research Corporation, New York, May 1981. 

b) The Vocational Expjoration Demonstration Project: A Preliminary 
Analysis of the 1979 Fall Component , Center for Urban Programs, 
St* Louis University, August 1980- 

c) Progress Toward Integrating Services for Youth , a series of 
quarterly reports on CYEP demonstrations. Technical Assistance 
and Training Corporation, Washington, D,C., January 1981. 



Competencies, Benchmarks, Assessment: Building Employability Programs 
That Meet Local Needs , Northwest Regional Education Laboratories, June 1981, 
pp. 14-15* 

^Ibid., pp. 14-15. 

^See: Robert J. Tolsma, Sharon E. Kahn, Stephen E. Marks, Carl H. Chiko, 
"A Model for Generating Competencies for the Employment Counselor," 
Journal of Employment Counseling , Vol. 16. No. 1, March 1979, pp. 5-15. 

^Ibid., p. 9. 



'See: Robert W. Brown and Jeffrey A. Kottler, "Increasing Client Employa- 
bility Through Skill Development, Journal of Employment Counsieling , 
Vol. 16, No* 3, September 1979, pp. 165-171. 

Competencies, Benchmarks, Assessment: Building Employability Programs 
That Meet Local Needs , op. cit. 

Northwest Labs concentrated quite a bit of this publication 
on identifying and developing assessment instruments and 
testing procedures. Several lists appear in the appendix 
to their work which identify publishers and specific testing 
instruments. 



^°Ibid., p. 39. 

^^Benchmarking: A State-of-the-Art Review, op. cit., pp. 71-73. 



"Ben A. Hirst, Jr., "The Components of Competency-Based Testing," op. cit., 
pp. 32-35. 

'pifth Annual Report of V-TECS , Vocational-Technical Education Consortium 
of States, Atlanta, Georgia, July 1978. 
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Appendix I 

VOCATIONAL EDUCATION: PROCESSES AND SCOPE 



A* ^l^rocesses 

In our research, we discovered several articles which described pro- 
cedures used to develop competency-based voc-ed programs and materials. 
Since these reflect practices developed after years of experience, they 
can serve as models for the Employment and Training System in efforts to 
develop similar approaches. 

In an American Vocational Journal article entitled, "The Components 

o'^ a Competency-Based Vocational Education," Ben Hirst suggests that 

there are ten basic components in developing a competenqy-based voc-ed 

program. In-essence, these components include the following: 

Component One: Employment Opportunities for Students . Job 
opportunities and future manpower needs are the primary fac- 
tors that determine whether a program is needed. Local labor 
market data are available to teachers from individual busi- 
nesses and industries, the local Department of Labor offices, 
or from the state division of vocational education. 

Component Two: Identifying Tasks That Workers Perform . Based 
on the information from Component One, jobs are identified for 
which students are pr-epared. A careful analysis is made to 
determine the tasks which workers employed in those jobs are 
required to perform. 

Component Three: Obtaining or Developing Occupational inventories . 
After developing the list of tasks which workers are required to 
perform, one should conduct interviews with workers who are actu- 
ally doing the jobs to verify and updat? the task list. Based on 
the verified task list, it should be possible to determine the 
degree of difficulty of each task, the time it takes to complete 
each task, and identify who performs the task. ^ 

Component Four: Analyzing and Using Occupational Survey Data . 
This step consists of arranging tasks according to the percentage 
of workers performing them, relative time spent and relative task 
difficulty. Before writing a performance objective, each task 
-must be analyzed to determine the steps the learner must follow 
to accomplish the task. The performance objective should be 
validated with a small group of employed workers. 
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Component Five: Analysis of Existing Materials and Media , 
The task statements* collection of task information, and 
the performance 'objectives serve as the basis for analyz- 
ing instructional materials and media* 

Component Six: Develop Needed Materials and Media , The 
task analysis ^may identify new units of work being accom- 
plished on the job. Before developing materials, determine 
whether you need informational materials, materials that 
show examples, or materials that provide learners with 
practice for meeting the criteria for" success. 

Component Seven: Developing Lesson Plans - The lesson 
plan is a specific plan defining the performance needed 
to transfer knowledge, direct the acquisition of knowledge, 
and demonstrate and g^jide student performance. The lesson 
plan is a blueprint for teacher performance and student 
learning. 

Component Eight: Test New Materials and Media and Lesson 
Plan . In a competency-based program, learning is the con- 
stant and time the variable. Instruction is provided until 
all of the students meet the performance criteria. Trying 
out materials, determining their effectiveness, and revis- 
ing materials until they meet the criteria for success are 
the key points in instructional validation. 

Component Nine: Revising Materials and Media . Instruc- 
tional materials should be reviewed periodically to deter- 
mine the need for revisions. New or revised materiel! s may 
be required to meet new performance criteria; changes in 
instructional focus, etc. 
• ^ 
Component Ten: Reviewing and Updating the Task Analysis . 
The occupational inventory , must undergo periodic review to 
(jetermine task changes in the world of work. Some occupa- 
tions change more frequently. The important rule is: keep 
your instructional program responsive to changes in the job. 

The V-4ECS model offers a good working example of how performance 
objectives. and performance guides for students in vocational-technical 
education are developed for state, regional and national usage. Initi- 
ated in 1973, the approach specifies uniform methods and procedures for 
the member ftates and agencies to ensure that quality products will be 
developed and transported among the states. The V-TECS consortium allows 
its memoers to share research and development. costs, minimize duplication 
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of effort 1ti occupational analysis and curriculum development, share the 
cost of expensive computer equipment and technical personnel, promote, 
the concept of competency-based progranjs, .and reduce the cost of materials 
development through improved project management techniques. 

V-TECS' Vole is to develop-catalogs of performance objectives and 
performance guides on specific jobs. The V-TECS methodology includes 
eleven steps. 

Step One: Prioritized List 

Job titles or occupational areas are selcted from a prioritized 
list derived from the Dictionary of Occupational Title^ (4th 
edition) . 

Step* Two: Selection of Agency to Develop Catalog 

A memorandum agreement is arranged between V-TECS and the state 
agency developing the catalog in an occupational area. 

Step Three: Preliminary Resea.rch , 

State-of-the-art research activities are conducted to identify 
performance-based curriculum materials and other relltted in- 
. formation. A formal report is prepared which includes the fol- 
lowing information: ^ > 

1. An assessment of existing curriculum material^. 

2. Available lists of tasks performed by workers! 

3. Tools and equipment used by workers. 

4. Sources for the identification of a population of 
workers . 

Step Four: Occupational .Inventory • 

The tasks, tools arYd equipment identified in STEP THREE are the 
♦ basis fotr developing the occupational Inventory. Workers are 
interviewed to"^estjiblish content validity. The inventory is 
sent to a predetermined sample of workers to establish construct 
validity. *The final inventory defines what is done, including 
the current tasks and procedures. 

Step Five: Population/Sample Design 

The workers involved with the job and the establishments where 
they work are identified. A 'Sample is drawn for survey purposes 
on the part of the agency developing the catalog. 

1^ 
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step Six: Report of Findings 

This report contains an updated state-of-the-art finding, an 
approved occupational inventory, a design for sample population, 
and a list of all references found. This report is sent to all 
members of the V-TECS consortium* 

Step. Seven: Data 

The data are derived froi||^he occupational survey, are computer 
processed and incorporated into the catalog development. 

Step Eight: Writing Team 

The writing team is made up of instructors with expertise in the 
occupational area, supervisors and incumbent workers. V-TECS 
trains the team. 

Step Nine: Field Review Proces.s 

The occupational survey data and the writing team materials are 
compiled into a field review document. The document is reviewed 
by instructional personnel, content specialists and curriculum 
development specialists. All of these inputs are evaluated and 
incorporated into the development of the final catalog. 

Step Ten: Final Catalog 
The final catalog contains: 

1. An overview of the developmental process, 

2. Duty and task statements listed from the occupa- 
tional inventory and writing team input, 

3. Performance objectives and performance guides for 
each ta^(<, 

4. A list of\tools/equipment, and 

5. A Cross-Reference Table which is an index of duties 
and tasks an,d performance objectives. 

\ 

The final catalog is ^ent^ to all V-TECS consortium members. 
Step Eleven: Dissemination 

A dissemination plan is required by V-TECS for each member state. 
Each state i.s responsible for getting the catalogs to the local 
programs. 

The V-TECS ca^ajogs are not in themselves curricula or performance- 
based programs. They serve as: 
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• A means for describing exactly those tasks which the 

' student can p^rl^orm and how well s/he can perform them; 

• A basis for developing criterion-referenced test items; 

• A means of identifying prerequisites such as minimum read- 
ing abilities, the ability to discriminate between colors 
and physical rec^ijirements, etc; 

• A rationale for jsequencing instruclnonal units; and 

/ 

• Intermediate learning checkpoints/ whiqh must be addressed 
in preparing the student for tasK accomplishment. 

The catalogs are stored in computers andj offer standards for job per- 
formance. The significance of these materials is the validation process 
and the procedural approach for adaptation in local programs. 

! . I I 

I 

B. Scope of the Vocational Educatioh System 

' Vocational ecfucation is an^'ntegral p'art of American education. Its 
intent is to prepare persons fpr entry into occupations; its goal/s are 
consistent v/ith the mor^ genei^ally stated} aims of formal education to pre- 
pare persons for meaningful and ^reductive lives. Vocational eflucation 
both complements and builds on basic and ^^pplied skills emphasized in the 
early years of fonr^l schooling. 

The Vocational Education Act (VEA) has\mandated a .reporting procedure, 
the Vocational Education Data System (VEDS),\to gather inforniation on train- 



ing programs and follow-up studies of program , lea vers who were trained in 
programs administered under the VEA. The char\s and analysis which fol- 
low summarize some of the data collected by VE^J^S for the year 1978-1979. 



/ 
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Vocational education programs enroll students at both the secondary 
and post-secondary levels. Figure 1 provicjes a breakdown of enrollments 
-at -these -two levels. 



Figure 1 


Vocational Education Program 




Enrollment Figures - 1973-79 




Secondary Enrollment 


12,765,080 


Post-secondary Enrollment 


6,820,064 


Total 


19,585,144 



This total represents enrollees in all programs, including vocational 
programs in most privately controlled institucions which are not included 
in State Plans for Vocational Education administered under the Vocational 
Education Act in 1978-79. 
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In the 1978-79 fiscal year 17,033,620 individuals were enrolled in 
VEA administered programs,. There were 27,753 provider institutions for 
these programs. Figure 2 is an analysis of the provider institutions. 

Figure 2 

ProvidefS of VocationaJ Education 
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The secondary providers total 17,710, or 63 ,8 percent of the total 
providers. The post-secondary providers total is 10,043, or 36-2 percent- 
Vocational education programs are traditionally grouped into nine 
program areas. An analysis of these occupational ly specific programs is 
included in Figure 3, 

Figure 3 

EnrollnMna in Voctdonai Education 
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The occupational ly specific enrollments total 7,625,937 or 44.7 
percent of the total VEA enrollment of 17,033,620. 

The distribution of enrollments in program areas by sex reveals 
considerable imbalance. Figure 4 is an analysis by program areas. 

Figure 4 

Sex Dteibution of Vocationii Studeott^by Program Arci 
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In summary, the purpose of vocational-technical education is to 
teach employment-related skills with a career focus. Vocational educa- 
tors have devised approaches for defining occupational inventories, 
derived from survey data, and validation procedures which include inter- 
views of on-the-job workers. Competency-based vocational education 
offers training programs in which performance objectives are used to 
specify the functional skills that are required for jobs in the world 
of work. These programs are designed to train individuals to a minimum 
level of skill proficiency, thus meeting the standards an employer has 
set for hiring individuals in an occupational area. 
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Appendix II 



Alterationist 

Auto Body 

Auto Mechanics 

Auto Parts Clerk 

Bank Teller 

Bookkeeper 

Carpenter 

Cashier/Checker 

Child Care 

Combination Welder 

Computer Prograniner 

Cosmetologist 

Cotton Ginning 

Data Processing 

Dental Assistant 

Emergency Med. Technician 

Floriculture 

Food Services 

Gardening-Grounds 

Home Furnishing 

Hospital Ward Clerk 



Hotel /Motel Management 
Housing Manager 
Industrial Sewing 
Legal Secretary 
Lie. Practical Nurse 
Machine Shop 
Masonry 
Nurseryman 
Nursing Assistant 
" Plumbing 
Printing Occupations 
Radio/TV Services 
Secretary 
Ship Operations 
Small Engine Repair 
Tax Collector 
Textile Production 
Timber Harvesting 
Tractor Mechanics 
Turf Management 
Word Processing/ 
Correspondence Sped al i st 



V-TECS Catalog Index - 1978 
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OCCUPATION: WORD PROCESSING/CORRESPONDENCE SPECIALIST 



DUTY: Supervising and Implementing 
TASK: Demonstrate Equipment Use 

Performance Objective 

Given a piece of equipment, its operations manual, and participants 
to view the demonstration; demonstrate the use of the equipment so that 
each feature of the equipment is demonstrated in accordance with opera- 
tion manual instructions and so that no safety regulations are violated. 

Performance Guide 

1. Review the operations manual. 

2. Test run equipment. 

3. Determine appropriate time and location for demon- ' 
stration and notify participants. 

4. Assemble required equipment, materials and supplies. 

5. Name the parts of the equipment for the participants. 

6. Demonstrate the safe use of each feature. 

7. Discuss safety precautions pertaining to normal 
operating procedures. 

8. Demonstrate the complete sequence of steps in per- 
formance of typical operations. 

9. Discuss the use of the operation manual for unusual 
procedures or troubleshooting strategies. 

10. Answer any questions. 

V-TECS Catalog of Tasks, Performance Objectives, 
Performance Guides, Tools, and Equipment, Page 22 



ERIC 



A. L. NELLUM AND ASSOCIATES -69- 



77 




OCCUPATION;^ WORD PROCESSING/CORRESPONDENCE SPECIALIST 
DUTY: Performing Clerical Activities 

TASK: Establish Alphabetical Filing System 

Performance Objective 

Given materials classified as to subject/name, manila folders, 
labels, A-Z file guides, a typewriter and a file drawer; establish an 
alphabetical filing system. The subject/name will be placed on each 
label and the folders will be placed in the drawer behind the appro- 
priate file guide. 

Performance Guide 

1. Assemble items to be filed. 

2. Review documents. 

3. Label manila folders as to subject name. 

4. Insert A-Z file guides in file drawer. 

5. File folders alphabetically. 



V-TECS Catalog of Tasks, Perfonnance Objectives, 
Performance Guides, Tools, and Equipment, Page 16 
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Appendix III 



ILLUSTRATIONS OF CYEP COMPETENCY AREAS, COMPETENCY INDICATORS 
ASSESSMENT METHODS AND BENCHMARKS 



■ i ■ 

COMPETENCY AREA 


COMPETENCY 
INDICAT6R 


ASSESSMENT 


BENCHMARK(S) 


Pre-Employment 


Knowledge of 
work oriented 
abilities, in- 
terests, values, 
and character- 
istics 


Assessment 
tests, diagnos- 
tic tests, 
exercisers 


Complete (accurate) 
self-assessment 
ranking from most 
to least those 
characteristics 
that are occupa- 
tional ly relevant 


Work Maturity 


Ability to 
carry out 
instructions 


Supervisor ' s 
monthly report 


Participant will 
carry out instruc- 
tions in an ex- 
peditious manner 


Educational 
Skills 


Word usage 


' Weekly testing 


Participant will 
be able to demon- 
strate the ability 

^ W 1 Km \m V» # 1 V» ^ •rill 

to: determine 
singular and plural 
nouns; identify 
double negatives; 
correctly use 
synonyms and anto- 
nyms; correctly 
form and use pos- 
sessives in both 
nouns and pronouns 


Occupational 
Skills (TV 
Installer) 


Mathematics 
for cable TV 

• 


Written test 


Student will 
answer, with 70 
percent accuracy, 
a test which in- 
cludes mathe- 
matical computa- 
tions performed 
as a cable TV 
installation 
technician 
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Appendix IV 

Sample Benchmarks for VEDP Participant Assessment 

.The St. Louis University Center for Urban Programs was contracted by 
the Qffice of Community Youth Employment Programs to conduct an assessment 
of the Vocational Exploration Demonstration Project (VEOP). 0.ne of several 
products developed by the Center for Urban Programs is a benchmark assess- 
ment manual which defines the purpose of each particular benchmark, describes 
the procedure for assjessing the achievement of that benchmark and outlines 
the procedure for recording the achievement. Appendix IV provides some sample 
benchmarks for selected competency indicators. 



Competency 
Area \ 


Competency 
Indicator 


Assessment 


Benchmarks 


Pre-employi^ent 


Interviewing 
•skills 


After receiving 
X hours of instruc- 
tion in employment 
interviewing tech- 
niques, the partici- 
oant win ao throuah 
a mock interview- 


• Was well-groomed 
t Was well-dressed 

• Began interview with 
friendly greeting 

Q Asked open-ended 
questions 


\ 


\ 




• Asked unbiased 
questions 

• Asked appropriate 

an relevant questions 
about duties and 
responsibilities 

• Listened attentively 
to responses 

• Maintained good eye | 
contact 1 


Pre-employment 

> 
If 

i 

> 

1 


Ability to 
obtain infor- 
mation about 
selected 
occupations 


After receiving 
X hours of instruc- 
tion in job search 
techniques, partici- 
' pant will demonstrate 
job search skills by 
researching employ- 
ment potential in 
three occupations 
and completing all 
of the tasks assigned 
by the instructor. 


• Identified a specific 
career field based on 
results of a job-related 
interest inventory 

• Identified three occupa- 
tions located in the 
above field 

• Identified the responsi- 
bilities and duties of 
the three occupations 
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Competency 
Area 


Contpetency 
Indicator 


Assessment 


Benchmarks 

* 


i 


/J 


* 


• Completed a self- 
assessment of employ- 
ment-related personal 
strengths and weaknesses • ' 

• Developed a list of 
questions, to ask an - / 
employer / ^ ^ 

• Obtained from one 
employer a description 

of the training experience-, 
resp6n5.ibil ities and 
duties of one of the 
occupations 


Pre-employment 


Ability to 
complete a 
job appli-ca- 
tion 


After receiving 
X hours of instruc- 
tion in completing 
a job application, 
participant^ wil,l 
complete a /written 
application form 
with 100% /accuracy, 

i 

1 

/ 


• Information has been 
•placed on correct lines 

• All information items 

have been completed • 

• All words are spelled 
correctly 

t Application is filled 
. out neatly 


Work 

maturity 


Attendance 


^— 

After receiving 
written instruction 
regarding attendance 
on the job, partici- 
pant performance will 
be evaluated by, work-, 
site supervisor/ 


• Has been in attendance 
100% of the time 

t Has been on time 100% 
of the days* 

^ 4- 


Work 

maturity 


Work-relevant 
behavior 


After receiving 
written instruction 
regarding appropriate 
behavior on the job, 
participant perfor- 
mance will be evalua- 
ted by work-site 
supervisor. 


f Has completed assigned 
tasks on a timely basis 

• Has responded to assigned 
•tasKs in a proper manner" 

• Has accurately completed 
assigned tasl^s 

• Has avoided inappropriate- 
behavior as defined by 

pol icies 

• Has maintained appropriate 
interactions in inter- 
personal situations 
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